40 research outputs found
Mitigating Interference in Content Delivery Networks by Spatial Signal Alignment: The Approach of Shot-Noise Ratio
Multimedia content especially videos is expected to dominate data traffic in
next-generation mobile networks. Caching popular content at the network edge
has emerged to be a solution for low-latency content delivery. Compared with
the traditional wireless communication, content delivery has a key
characteristic that many signals coexisting in the air carry identical popular
content. They, however, can interfere with each other at a receiver if their
modulation-and-coding (MAC) schemes are adapted to individual channels
following the classic approach. To address this issue, we present a novel idea
of content adaptive MAC (CAMAC) where adapting MAC schemes to content ensures
that all signals carry identical content are encoded using an identical MAC
scheme, achieving spatial MAC alignment. Consequently, interference can be
harnessed as signals, to improve the reliability of wireless delivery. In the
remaining part of the paper, we focus on quantifying the gain CAMAC can bring
to a content-delivery network using a stochastic-geometry model. Specifically,
content helpers are distributed as a Poisson point process, each of which
transmits a file from a content database based on a given popularity
distribution. It is discovered that the successful content-delivery probability
is closely related to the distribution of the ratio of two independent shot
noise processes, named a shot-noise ratio. The distribution itself is an open
mathematical problem that we tackle in this work. Using stable-distribution
theory and tools from stochastic geometry, the distribution function is derived
in closed form. Extending the result in the context of content-delivery
networks with CAMAC yields the content-delivery probability in different closed
forms. In addition, the gain in the probability due to CAMAC is shown to grow
with the level of skewness in the content popularity distribution.Comment: 32 pages, to appear in IEEE Trans. on Wireless Communicatio
Wireless Data Acquisition for Edge Learning: Data-Importance Aware Retransmission
By deploying machine-learning algorithms at the network edge, edge learning
can leverage the enormous real-time data generated by billions of mobile
devices to train AI models, which enable intelligent mobile applications. In
this emerging research area, one key direction is to efficiently utilize radio
resources for wireless data acquisition to minimize the latency of executing a
learning task at an edge server. Along this direction, we consider the specific
problem of retransmission decision in each communication round to ensure both
reliability and quantity of those training data for accelerating model
convergence. To solve the problem, a new retransmission protocol called
data-importance aware automatic-repeat-request (importance ARQ) is proposed.
Unlike the classic ARQ focusing merely on reliability, importance ARQ
selectively retransmits a data sample based on its uncertainty which helps
learning and can be measured using the model under training. Underpinning the
proposed protocol is a derived elegant communication-learning relation between
two corresponding metrics, i.e., signal-to-noise ratio (SNR) and data
uncertainty. This relation facilitates the design of a simple threshold based
policy for importance ARQ. The policy is first derived based on the classic
classifier model of support vector machine (SVM), where the uncertainty of a
data sample is measured by its distance to the decision boundary. The policy is
then extended to the more complex model of convolutional neural networks (CNN)
where data uncertainty is measured by entropy. Extensive experiments have been
conducted for both the SVM and CNN using real datasets with balanced and
imbalanced distributions. Experimental results demonstrate that importance ARQ
effectively copes with channel fading and noise in wireless data acquisition to
achieve faster model convergence than the conventional channel-aware ARQ.Comment: This is an updated version: 1) extension to general classifiers; 2)
consideration of imbalanced classification in the experiments. Submitted to
IEEE Journal for possible publicatio
Privacy For Free:Wireless Federated Learning Via Uncoded Transmission With Adaptive Power Control
Federated Learning (FL) refers to distributed protocols that avoid direct raw
data exchange among the participating devices while training for a common
learning task. This way, FL can potentially reduce the information on the local
data sets that is leaked via communications. In order to provide formal privacy
guarantees, however, it is generally necessary to put in place additional
masking mechanisms. When FL is implemented in wireless systems via uncoded
transmission, the channel noise can directly act as a privacy-inducing
mechanism. This paper demonstrates that, as long as the privacy constraint
level, measured via differential privacy (DP), is below a threshold that
decreases with the signal-to-noise ratio (SNR), uncoded transmission achieves
privacy "for free", i.e., without affecting the learning performance. More
generally, this work studies adaptive power allocation (PA) for decentralized
gradient descent in wireless FL with the aim of minimizing the learning
optimality gap under privacy and power constraints. Both orthogonal multiple
access (OMA) and non-orthogonal multiple access (NOMA) transmission with
"over-the-air-computing" are studied, and solutions are obtained in closed form
for an offline optimization setting. Furthermore, heuristic online methods are
proposed that leverage iterative one-step-ahead optimization. The importance of
dynamic PA and the potential benefits of NOMA versus OMA are demonstrated
through extensive simulations.Comment: Published in IEEE Journal on Selected Areas in Communications (
Volume: 39, Issue: 1, Jan. 2021) DOI: 10.1109/JSAC.2020.303694
Wireless federated Langevin Monte Carlo: repurposing channel noise for Bayesian sampling and privacy
Most works on federated learning (FL) focus on the most common frequentist formulation of learning whereby the goal is minimizing the global empirical loss. Frequentist learning, however, is known to be problematic in the regime of limited data as it fails to quantify epistemic uncertainty in prediction. Bayesian learning provides a principled solution to this problem by shifting the optimization domain to the space of distribution in the model parameters. This paper proposes a novel mechanism for the efficient implementation of Bayesian learning in wireless systems. Specifically, we focus on a standard gradient-based Markov Chain Monte Carlo (MCMC) method, namely Langevin Monte Carlo (LMC), and we introduce a novel protocol, termed Wireless Federated LMC (WFLMC), that is able to repurpose channel noise for the double role of seed randomness for MCMC sampling and of privacy preservation. To this end, based on the analysis of the Wasserstein distance between sample distribution and global posterior distribution under privacy and power constraints, we introduce a power allocation strategy as the solution of a convex program. The analysis identifies distinct operating regimes in which the performance of the system is power-limited, privacy-limited, or limited by the requirement of MCMC sampling. Both analytical and simulation results demonstrate that, if the channel noise is properly accounted for under suitable conditions, it can be fully repurposed for both MCMC sampling and privacy preservation, obtaining the same performance as in an ideal communication setting that is not subject to privacy constraints
Bayesian Over-the-Air FedAvg via Channel Driven Stochastic Gradient Langevin Dynamics
The recent development of scalable Bayesian inference methods has renewed
interest in the adoption of Bayesian learning as an alternative to conventional
frequentist learning that offers improved model calibration via uncertainty
quantification. Recently, federated averaging Langevin dynamics (FALD) was
introduced as a variant of federated averaging that can efficiently implement
distributed Bayesian learning in the presence of noiseless communications. In
this paper, we propose wireless FALD (WFALD), a novel protocol that realizes
FALD in wireless systems by integrating over-the-air computation and
channel-driven sampling for Monte Carlo updates. Unlike prior work on wireless
Bayesian learning, WFALD enables (\emph{i}) multiple local updates between
communication rounds; and (\emph{ii}) stochastic gradients computed by
mini-batch. A convergence analysis is presented in terms of the 2-Wasserstein
distance between the samples produced by WFALD and the targeted global
posterior distribution. Analysis and experiments show that, when the
signal-to-noise ratio is sufficiently large, channel noise can be fully
repurposed for Monte Carlo sampling, thus entailing no loss in performance.Comment: 6 pages, 4 figures, 26 references, submitte
Wireless Federated Langevin Monte Carlo: Repurposing Channel Noise for Bayesian Sampling and Privacy
Most works on federated learning (FL) focus on the most common frequentist
formulation of learning whereby the goal is minimizing the global empirical
loss. Frequentist learning, however, is known to be problematic in the regime
of limited data as it fails to quantify epistemic uncertainty in prediction.
Bayesian learning provides a principled solution to this problem by shifting
the optimization domain to the space of distribution in the model parameters.
{\color{black}This paper proposes a novel mechanism for the efficient
implementation of Bayesian learning in wireless systems. Specifically, we focus
on a standard gradient-based Markov Chain Monte Carlo (MCMC) method, namely
Langevin Monte Carlo (LMC), and we introduce a novel protocol, termed Wireless
Federated LMC (WFLMC), that is able to repurpose channel noise for the double
role of seed randomness for MCMC sampling and of privacy preservation.} To this
end, based on the analysis of the Wasserstein distance between sample
distribution and global posterior distribution under privacy and power
constraints, we introduce a power allocation strategy as the solution of a
convex program. The analysis identifies distinct operating regimes in which the
performance of the system is power-limited, privacy-limited, or limited by the
requirement of MCMC sampling. Both analytical and simulation results
demonstrate that, if the channel noise is properly accounted for under suitable
conditions, it can be fully repurposed for both MCMC sampling and privacy
preservation, obtaining the same performance as in an ideal communication
setting that is not subject to privacy constraints.Comment: submitte
Leveraging Channel Noise for Sampling and Privacy via Quantized Federated Langevin Monte Carlo
For engineering applications of artificial intelligence, Bayesian learning
holds significant advantages over standard frequentist learning, including the
capacity to quantify uncertainty. Langevin Monte Carlo (LMC) is an efficient
gradient-based approximate Bayesian learning strategy that aims at producing
samples drawn from the posterior distribution of the model parameters. Prior
work focused on a distributed implementation of LMC over a multi-access
wireless channel via analog modulation. In contrast, this paper proposes
quantized federated LMC (FLMC), which integrates one-bit stochastic
quantization of the local gradients with channel-driven sampling.
Channel-driven sampling leverages channel noise for the purpose of contributing
to Monte Carlo sampling, while also serving the role of privacy mechanism.
Analog and digital implementations of wireless LMC are compared as a function
of differential privacy (DP) requirements, revealing the advantages of the
latter at sufficiently high signal-to-noise ratio.Comment: 5 pages, 4 figures, submitte